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ABSTRACT 


We  consider  a  fractional  flow  nodal  of  a  graded  manpower  system 
and  develop  algorithms  for  calculating  optimal  control  policies 
in  four  situations:  (1)  finite  time  horizons  with  no  constraints 
on  staff  distributions,  (ii)  finite  time  horizon  with  constraints 
on  final  staff  distribution,  (ill)  infinlts  horizon  with  con¬ 
straints  on  staff  distribution  aai  (iv)  problems  with  a  nonsta¬ 
tionary  transient  stage  and  an  Infinite  stationary  stage.  In 
each  case  results  developed  in  solving  the  simpler  problems  are 
useful  in  analyzing  more  complicated  situations. 

In  addition  to  providing  computational  procedures  we  apply  the 
algorithms  to  a  three  rank  model  and  discuss  the  possible  uses 
and  limitations  of  our  procedure. 


INTRODUCTION 


This  paper  studies  a  fractional  flow  model  of  a  graded  manpower  system. 

A  useful  mathematical  programming  procedure  for  evaluation  of  alternate  policies 
is  presented,  and  some  illustrative  examples  given. 

The  model  is  designed  as  an  aggregate  planning  device;  it  is  not  designed 
to  specify  hiring  and  promotion  policy  in  an  exact  sense,  but  to  assist  in 
answering  questions  of  the  following  sort:  (i)  How  should  the  growth  of  a  new 
organization  be  scheduled?  (ii)  What  are  the  costs  of  institutional  restric¬ 
tions  on  staff?  (iii)  How  would  the  system  react  to  a  change  in  promotion 
policy?  (lv)  What  is  the  relation  between  the  cost  of  operation  and  the  rate 
of  growth?  (v)  Whet  impact  would  a  wage  Increase  have  on  the  cost  of  future 
operations? 

An  extended  discussion  of  similar  models  in  manpower  planning  can  be  found 
in  Bartholomew  [1].  This  paper  follows  Bartholomew  [2]  in  the  basic  model  struc¬ 
ture  and  search  for  optimal  pollclee. 

Although  models  of  this  type  are  often  called  Harkov  models,  we  believe 
this  name  is  misleading.  The  Markov  interpretation  implies  there  is  a  stochastic 
decision  rule  that  governs  promotion  policy  for  each  individual  in  the  system. 

We  must  either  assume  that  all  individuals  are  the  same  and  face  the  same  sto- 
chaetlc  promotion  mechanism  or  that  different  classes  of  individuals  face  dif¬ 
ferent  promotion  possibilities  and  that  the  given  promotion  matrix  can  describe 
the  aggregate  behavior  of  all  classes.  Both  of  these  assumptions  are  difficult 
to  defend.  In  addition,  if  the  coefficients  truly  represent  probabilities, 

then  the  variances  of  random  variables  such  as  tha  number  of  years  in  the  system, 
and  the  total  salary  received  should  be  meaningful.  Our  rough  calculations  have 
indicated  that  these  variances  are  far  too  large  to  be  consistent  with  observed 
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behavior.  Therefore  we  prefer  the  deterministic  fractional  flow  interpretation 
of  the  model.  The  organization  aa  a  matter  of  policy  decides  to  promote  a 
fraction  of  people  in  rank  1  to  rank  2  each  year.  Our  model  is  designed  to 
explore  the  consequences  of  that  policy  decision. 

The  system  has  n  ranks  1,2,  ...»  n  .  In  period  t  the  n  vector 

x(t)  =  [x^(t),  ...,  xn(t)]  describes  the  distribution  of  manpower  among  the 
ranks.  We  assume  the  initial  distribution  x(0)  is  known.  An  n  x  n  pro¬ 
motion  matrix  P  governs  transitions  within  the  system,  where  P„  is  the 

fraction  of  workers  in  rank  i  who  move  to  rank  j  .  We  generally  assume 
that  P  is  independent  of  t  ,  This  assumption  is  crucial  for  the  infinite 

horizon  cases  in  Section  III  and  part  of  IV.  In  Sections  I  and  II,  however, 

P  can  vary  with  t  . 

The  system  hiring  policy  is  described  by  the  n-vector  u(t)  *  [u,(t),  ...,  u  (t)] 

in 

where  u^  (t)  is  the  number  of  new  appointments  in  rank  j  made  during  period  t  . 

We  shall  assume  throughout  that  u(t)  is  nonnegative:  there  is  no  firing.  We 
adopt  the  convention  that  these  new  workers  start  work  in  period  t  +  1  ,  there¬ 
fore 


n 

x  (t  +  1)  -  l  x.  (t)P..  +  u.(t)  . 
J  i-1  1  13  3 


In  matrix  notation 


(1)  x(t  +  1)  -  x(t)P  +  u(t)  . 

Equation  (1)  is  the  system's  Law  of  Motion. 

We  assume  the  system  grows  at  rate  (6-1)  .  We  have  expansion,  no  growth, 
or  contraction  as  8>1,  8*1,  or  0  <  1  .  Let  e  be  an  ti  dimension  column 


vector  with  each  component  equal  to  one.  At  time  t  the  growth  constraint  is 


x(t)e  =  8tx(0)e  for  t  j>  1  . 

Alternate  growth  constralnto  are  possible.  For  example,  let  f  be  an  n 
dimension  column  vector  with  equal  to  the  total  annual  institutional  finan¬ 

cial  support  given  to  a  worker  in  rank  i  .  Then  x(t)f  gives  the  total  man¬ 
power  budget  in  period  t  .  The  growth  constraint  could  also  read 

(2)  x(t)f  -  eScCOJf  for  t  >  1 

to  reflect  a  changing  manpower  budget.  A  strict  cost  interpretation  for  (2)  is 
not  necessary:  for  example  f^  could  represent  some  measure  of  the  productivity 
of  ranx  i  staff  members.  In  that  case,  condition  (2)  would  give  the  total  system 
productivity  requirement  for  the  tC^  period. 

In  the  optimization  models  described  below  our  objective  is  a  linear  function 
of  the  state  vector  x(t)  of  staff  distribution  and  the  control  vector  u(t)  of 
new  appointments.  Throughout  we  Interpret  this  objective  as  the  cost  of  system 
operations.  There  are  two  reasons  for  this:  first  it  eases  the  exposition, 
second  it  is  a  useful  criterion.  The  minimum  cost  of  system  operations  is  a 
lower  bound  on  the  cost  that  could  be  Incurred  using  any  other  policy.  It  is 
reasonable  to  assume  that  the  institution's  personnel  policy  will  depart  from 
the  minimum  cost  policy  indicated  by  our  model.  However,  knowledge  of  the  mini¬ 
mum  cost  policy  also  provides  a  direction  of  policy  change,  if  the  institution 
desires  to  change  policies  and  reduce  costs.  As  with  the  growth  constraint  (2), 
alternative  interpretations  of  the  linear  objective  are  allowable.  We  could, 
for  example,  use  the  objective  function  to  measure  productivity,  and  seek  to 
maximize  it  subject  to  budgetary  or  size  restrictions.  In  essence,  all  the 
results  in  this  paper  would  carry  over  in  this  case. 

In  Section  I  we  Introduce  a  finite  horizon  planning  problem  with  no  constraints 
on  the  staff  distribution  and  develop  an  efficient  dynamic  programming  procedure 
to  solve  this  problem.  Section  II  considers  a  slight  generalization  of  the  finite 


horizon  planning  problem:  constraints  are  placed  on  the  final  distribution 
x(T)  .  This  problem  is  solved  using  generalized  linear  programming,  where  the 
dynamic  programming  algorithm  described  in  Section  I  is  the  column  generating 
subproblem.  Section  III  considers  infinite  horizon  p  oblerns  and  computes  lower 
bounds  on  the  optimal  value.  We  obtain  asymptotically  optimal  stationary  linear 
decision  rules.  This  easily  implemented  policy  is  optimal  if  the  resulting  se¬ 
quence  of  staff  distributions  x(t)  does  not  violate  the  system  constraint.  In 
Section  IV  the  theory  of  Sections  II  and  III  are  blended  together  to  examine  an 
infinite  horizon  planning  problem  that  combines  a  T  year  transient  stage  and 
an  indefinite  stationary  stage.  As  in  Section  III  we  are  able  to  derive  lower 
bounds  on  the  optimal  value  of  this  two  stage  planning  problem. 

We  solved  the  problems  presented  in  Sections  I-IV  using  a  dynamic  program¬ 
ming  and  generalized  linear  programing.  All  of  these  problems  could  have  been 
attacked  directly  by  solving  larger  linear  programs.  We  took  the  dynamic  pro¬ 
gramming  generalized  LP  approach  for  several  reasons.  Our  analysis  of  the 
problem  developed  in  exactly  the  way  our  results  are  presented.  We  first  de¬ 
veloped  the  DP  procedure  to  study  the  free  end  point  problem,  and  then  found 
that  these  results  could  be  used  to  help  solve  the  more  complicated  problems 
with  terminal  constraints.  We  feel  that  the  generalized  LP  approach  is  a 
more  natural  way  to  analyze  the  problems  presented  in  Sections  II  and  IV  since 
it  focuses  attention  on  properties  of  sets  in  the  n  dimension  space  of  staff 
distributions.  In  addition,  the  dynamic  programming  procedure  is  efficient  and 
application  of  generalized  LP  is  new  and  of  independent  interest.  Our  compu¬ 
tations  were  done  interactively,  and  we  frequently  changed  the  data  and  policy 
environment  to  examine  the  effects  of  these  changes  on  appointment  policies  and 
staff  distributions.  The  compact  form  of  the  data  storage  needed  in  the  gener¬ 
alized  LP  approach  saved  us  from  regenerating  large  arrays  each  time  the  prob¬ 
lem  was  changed. 


We  did  not  test  each  method  to  see  which  was  numerically  more  efficient  on 
a  set  of  test  problems.  That  type  of  analysis  seemed  far  removed  from  the  pur¬ 
pose  of  the  paper.  We  were,  however,  satisfied  with  the  speed  and  storage  re¬ 
quirements  of  the  approach  we  selected. 

In  Section  V  we  examine  a  three  state  university  planning  model  and  give 
two  examples  of  how  the  theory  developed  in  Sections  I-1V  can  be  used.  The 

% 

example's  small  size  is  consistent  with  the  purpose  of  the  model.  The  organi¬ 
zation's  policy  is  as  much  determined  by  the  growth  factor  0  ,  and  the  promo¬ 
tion  matrix  P  as  the  hiring  policy  u(t)  .  A  small  model  allows  us  to  do 
extensive  sensitivity  analysis  to  determine  the  effects  of  changing  8  ,  P  , 
and  other  system  parameters.  A  reader  interested  in  the  model's  value  as  an 
aid  to  decisionmakers  can  turn  directly  to  Section  V  without  reading  further. 

The  following  definitions  and  notations  are  used.  The  symbol  £  means 
"is  defined  to  be".  Given  the  vector  f  with  f^  >  0  we  say 

S  -  (x(t)  |  x(t)f  -  0txf,x(t)  >  0} 

where  x  is  the  initial  distribution.  is  the  set  of  possible  distribution 

at  time  t  ,  assuming  a  growth  rate  of  (8  -  1)  in  each  period,  and  Sq  -  S  . 

Let  R(x)  «  {y  |  y  >  xP.yf  -  8xf}  .  Starting  at  x  ,  R(x)  is  the  set  of 
points  that  can  be  reached  in  one  period.  For  any  t  and  set  A  C  Sfc 

R(A)  -  u  R(x)  ■  (y  |  y  t  R(x),x  c  A}  . 
xcA 

R(A)  is  the  set  of  points  that  can  be  reached  in  one  period  from  the  set  A  . 
Let  R*(A)  ■  R(A)  ,  and  for  t  >  2  , 

RC (A)  -  R(Rt"1(A))  . 

Rt(A)  is  the  set  of  points  that  can  be  reached  from  A  in  exactly  t  periods. 
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Let 


E  -  {y  |  6y  >  yP,y  >  0}  . 

If  y  £  E  ,  then  x(t)  =  0fcy  is  a  sequence  that  obeys  the  size  constraint  (2) 
and  the  law  of  motion  (1)  with  u(t)  =  0y  -  yP  .  Note  that  E  =  {y  |  6y  e  R(y),y  _>  0 
In  Section  III  we  designate  the  promotion  matrix  as  Q  and  the  growth  factor 
as  6  .  This  allows  us  to  distinguish  the  transient  and  stationary  promotion  ma¬ 
trices  and  growth  rates  when  they  are  united  in  Section  IV.  We  denote  the  empty 
set  as  <(>  ,  and  use  cl  for  closure  and  ri  for  relative  interior. 
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I.  MINIMUM  COST  OPERATION:  FINITE  TIME 
HORIZON  -  NO  TERMINAL  CONSTRAINTS. 

This  section  develops  an  efficient  method  for  minimizing  the  cost  of 
operations  over  a  T  year  horizon  with  no  terminal  constraints. 

The  distribution  x(t)  follows  the  law  of  motion  described  in  the  intro¬ 
duction  with  the  quantity  x(t)f  growing  at  a  constant  rate  0  -  1  .  The  present 
value  of  cost  incurred  in  period  t  is  x(t)c(t)  -I-  u(t)d(t)  .  There  is  a 
reward  x(T)q  (perhaps  zero)  attached  to  the  terminal  distribution  x(T)  . 

The  present  value  of  all  costs  is  thus 

T-l 

l  x(t)c(t)  +  u(t)d (t)  -  x(T)q  . 
t-0 

We  assume  the  current  distribution  x(0)  -  x  is  known. 

The  problem  is 

T-l 

(1)  Minimize  £  x(t)c(t)  +  u(t)d(t)  -  x(T)q 

t-0 


Subject  to 


x(t  +  1)  -  x(t)P  +  u(t) 

x(t)f  -  eScf 

u(t)  >  0  t  —  0,1,2,  ...,  T-l 

x(0)  -  x  _>  0 

The  first  set  of  constraints  is  the  law  of  motion  that  describes  distribution 
changes  over  time.  The  second  set  of  constraints  indicates  that  the  quantity 
x(t)f  grows  at  a  constant  rate  (8-1)  .  The  third  set  of  constraints  says 
that  no  firing  takes  place. 


Note  that 
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x(t  +  l)f  =  x(t)Pf  +  u(t)f  *  0x(c)f 

Thus  u(t)f  »  x(t)[0I  -  Pjf  .  Let  v  =  (01  -  P]f  .  Our  solution  procedure 
below  a88Krrt39  v  ts  strictly  positive  and  P  ^  0  .  When  f  ■  e  ,  constraint 
u(t)e  ■  x(t)v  *  x(t)e(0  -  1)  +  x(t) (I  -  P)e  ,  says  the  number  of  people  hired 
is  equal  to  the  number  that  leave,  x(t)(I  -  P)e  ,  plus  the  number  needed  to 
increase  size,  (0  -  l)x(t)e  . 

Although  we  shall  not  work  out  the  details,  the  analysis  below  is  still 
valid  if  P  is  a  function  P(t)  of  time  and  v(t)  -  (01  -  P(t)]f  is  strictly 
positive  for  all  t  . 

Problem  (1)  can  be  solved  with  an  efficient  dynamic  programming  technique. 

For  s  »  0,1,2  ...  T  define  V(x,s)  to  be  the  optimal  value  of 

T-l 

(2)  Minimize  £  x(t)c(t)  +  u(t)d(t)  -  x(T)q 

t-s 

Subject  to  x(t  +  1)  *  x(t)P  +  u(t) 
u(t)f  -  x(t)v 

u(t)  >0  t  ■  s,s+l,  ...,  T-l 

x(s)  *  x  _>  0  given 

Note  that  V(x,0)  is  the  optimal  value  of  (1)  as  a  function  of  the 
initial  conditions  x(0)  -  x  and  that  V(x,T)  -  -xq  .  The  optimal  value  function 
obeys  the  usual  dynamic  programming  principle  of  optimality. 

(3)  V(x,t)  =*  xc(t)  +  Min  ud(t)  +  V(xP  +  u,t  +  1) 

uf “XV  L  J 

u>0 


Assume 


V  (x,t  +  1)  -  xh(t  +  1) 
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a  linear  function. 

For  t  ■  T  -  l,h(T)  ■  -q  .  Thus  this  assumption  is  valid  for  t  ”  T  -  1  . 
Then  equation  (3)  becomes 


(4) 


V(x,t)  •  x[c(t)  +  Ph(t  +  1)J  +  Min  [u(d(t)  +  h(t  +  1))] 

uf«xv 

u>0 


Since  v  >  0  ,  xv  is  strictly  positive  for  any  semi-positive  x  .  Thus  the 
linear  program  in  this  functional  equation  has  a  trivial  solution.  Let  tr(t) 
be  the  index  such  that 


(5) 


[d(t)  +  h(t  +  1)] 


ff(t) 


[d(t)  +  h(t  +  1)1 


»(t) 


Min 

i-l,2,...,N 


The  optimal  solution  to  the  linear  program  in  (4)  is 


j  0  if  i  +  «<t) 
11  (f^  if  i  -  »<t) 


Therefore 


V(x,t)  ■  x(c(t)  +  Ph(t  +  1))  +  xvn  . 


Set 

(6)  h(t)  ■  c(t)  +  Ph(t  +  1)  +  nv  . 

By  this  constructive  induction  we  have  demonstrated 

Theorem  1: 

The  optimal  solution  of  problem  (1)  is  to  set 

if  i  ■  *(t) 

0  otherwise 
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The  index  ir(t)  and  the  vector  h(t)  are  calculated  recursively  from  (5), 

(6),  and  the  initial  condition  h(T)  =  -q  . 

A  more  general  analysis  of  this  no  terminal  constraint  model  can  be  found 
in  Section  1  of  [7].  The  case  of  nondecreasing  size  is  treated  and  a  particu¬ 
lar  class  of  problems  is  shown  to  have  a  closed  form  solution.  However,  we  wish 
to  use  this  model  as  a  building  block  in  solving  more  detailed  problems  with 
terminal  constraints. 

It  is  possible,  by  eliminating  the  variables  x(t)  from  problem  (1),  to 
express  (1)  as  a  linear  program  with  T  equality  constraints  and  n  x  T  variables. 
In  choosing  the  dynamic  programming  approacn  to  the  solution  of  problem  (1),  we 
considered  several  possible  advantages  and  disadvantages  of  the  two  methods. 

First,  it  must  be  understood  that  we  were  doing  Interactive  computations.  The 
interactive  approach  is  most  useful  if  the  programs  allow  us  to  quickly  analyze 
the  effects  of  alternate  policies  or  data  lnputa.  In  other  words  we  could  change 
P  ,  f  ,  c  ,  d  ,  x(0)  ,  and  q  easily  and  resolve  the  problem.  The  dynamic  pro¬ 
gram  offered  greater  flexibility  in  changing  the  data  and  it  also  had  a  much 
smaller  storage  requirement.  Both  of  these  qualities  are  important  in  inter¬ 
active  computations.  The  sole  disadvantage  we  saw  with  the  dynamic  programming 
approach  was  the  necessity  to  code  a  special  algorithm.  In  contrast,  interactive 
LP  codes  are  already  available.  However,  to  analyze  a  model  Interactively  using 
linear  programming  it  is  necessary  to  write  a  matrix  generation  program  that  con¬ 
structs  the  LP  constraint  matrix  from  the  problem's  data  and  then  retranslates 
the  LP  solution  into  the  terms  of  the  original  problem.  We  found  it  was  as  easy 
to  write  the  dynamic  programming  code  as  to  construct  the  matrix  generator  for 
the  LP.  In  addition,  it  seemed  unlikely  to  us  that  an  LP  could  improve  on  the 
extremely  simple  calculations  required  in  (5)  and  (6).  Finally,  we  shall  in 
later  sections  be  faced  with  similar  choices  in  solution  strategies  for  more 
complicated  problems.  In  those  situations  we  believed  that  the  dynamic  programming 


approach  offered  even  greater  advantages.  Thus  it  was  necessary  to  write  the 
dynamic  programming  code  for  use  as  a  subroutine  in  the  more  complicated  prob¬ 
lems. 
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II.  MINIMAL  COST  OPERATION:  A  FINITE  HORIZON 
AND  A  TERMINAL  CONSTRAINT. 

This  section  refines  the  model  presented  in  Section  i  by  adding  the  terminal 
constraint,  x(T)  t  C  -  (y  |  yA  >  0)  .  Since  S^  =  {y  |  y  =  0  x£,y  >  0}  ,  the 
terminal  constraint  can  also  be  written  as 

x(T)  c  C  n  sT  -  B  . 

Any  polyhedral  subset  B  of  S^  can  be  represented  in  the  manner  described 
above. 

As  one  example  consider  a  restriction  that  the  fraction  of  workers  in  ranks 
h  +  l,h  +  2,  . ..,  N  is  less  than  6  .  This  can  be  written 

h  N 

«  l  x,(T)  +  (d  -  1)  l  x  (T)  >  0  . 
i-1  *  i-h+1  x 

We  can  also  specify  final  target  distribution  x(T)  . 

If  there  are  K  terminal  constraints  then  A  is  an  N  *  K  matrix. 

The  optimization  problem  is 

T-l 

(1)  Minimize  £  x(t)c(t)  +  u(t)d(t) 

t-0 

Subject  to  x(t  +  1)  -  x(t)P  +  u(t) 
u(t)f  ■  x(t)v 

u(t)  >0  t  -  0,1,  ...,  T-l 
x(T)A  >  0 

x(0)  ■  x  ^  0  given. 


This  is  exactly  problem  (1)  of  Section  1  with  q  »  0  ,  and  an  additional  constraint 
x(T)A  >  0  . 

T 

Recall  from  the  introduction  that  R  (x)  is  the  set  of  all  points  that 

T 

can  be  reached  from  x  in  exactly  T  steps.  If  R  (x)  fl  C  ■  4  »  then  problem 

(1)  is  infeasible:  there  is  no  x(T)  satisfying  the  terminal  constraint. 

T 

For  each  y  e  R  (x)  define  W(y)  as  the  optimal  value  of 

T-l 

Minimum  £  x(t)c(t)  +  u(t)d(t) 
t-0 

Subject  to  x(t  +  1)  -  x(t)P  +  u(t) 
u(t)f  -  x(t) V 
u(t)  >  0 
x(T)  -  y 
x(0)  -  x  >  0 

T 

U(y)  is  the  minimum  cost  of  going  from  x  to  y  in  T  steps.  For  y  i  R  (x) 
va  define  W(y)  ■  +»  .  Problem  (1)  is  equivalent  to 

(2)  Minimize  W(y) 

Subject  to  y  e  C  fl  RT(x) 

Thle  section  shows  how  we  can  exploit  the  efficient  dynamic  programming 
procedure  developed  in  Section  I  to  solve  the  problem  with  a  terminal  constraint. 
The  method  is  outlined  below  using  Dantzlg  and  Wolfe's  generalized  linear 
programming  [4].  We  maintain  a  feasible  solution  of  problem  (2)  at  all  times 
and  calculate  a  sequence  of  lower  bounds  on  the  optimal  value  of  (2) . 

After  describing  the  algorithm  we  will  show  how  an  optimal  solution  of  (1) 
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is  constructed  from  an  optimal  solution  of  (2).  Then  we  give  a  theoretical 
justification  of  the  algorithm  and  we  present  a  phase  I  procedure  that  will 

either  initialize  the  algorithm  or  show  that  problem  (2)  is  infeasible. 

X  2  id  T  X 

Let  y  ,y  i  ....  y  be  points  in  R  (x)  ,  and  let  «  W (y1)  . 

The  master  linear  program  is 

m 

(3)  Minimize  l  AW 

i-1  1  i 

Subject  to  J  A . (y*A)  >  0 

i-1  1 


m 


l 

i-1 


-  1 


A  >  0 

m 

Let  Am  be  the  optimal  solution  of  (3).  Then  £  A“y  is  feasible  for  (2) 

i-1  1  1 


m 

and  £  A™W,  is  an  upper  bound  on  the  optimal  value  of  (2) . 
i-1 

Let  (r^.o111)  be  the  optimal  dual  variables  associated  with  problem  (3). 
m 

Note  that  om  -  l  A°W  .  The  eubproblem  is 
i-1 

(4)  Minimize  W(y)  -  yArm 


y  e  RT(x)  . 


Subproblem  (4)  is  solved  by  setting  qm  -  Arm  and  solving  problem  (1)  of 

Art"!  m  1 2. 

Section  1.  Let  y  be  the  optimal  solution  of  (4)  and  let  v  be  the 

optimal  value  of  (4).  Then  v  is  a  lower  bound  on  the  optimal  value  of  (2) 


and 


We  consider  two  cases: 


1.  If 


<  a 


2.  If 


of  (2)  is 


then  column  (W^^.y  A)  is  added  to  problem  (3). 
m 

then  £  ATy  is  optimal  for  (2)  and  the  optimal  value 
i-1 


The  generalised  linear  programming  algorithm  solves  (3)  and  (4)  while 
successively  generating  a  sequence  of  upper  and  lower  bounds  on  the  optimal 

m  m  i 

value.  If  optimal  termination  occurs  on  iteration  m  ,  then  y  -  £  A™y 

i-1  1 

is  the  optimal  solution  of  (2). 

After  we  have  discovered  the  optimal  solution  y  of  problem  (2)  we  need 
to  reconstruct  jx(t)  ,u(t)j  ,  the  optimal  solution  of  problem  (1).  Let 


J  -  {j|x“  >  °j  .  Then  y  - 
policy  used  to  generate  y^ 


l  A^y1  .  For 
J  2 

.  To  go  from 


j  e  J  ,  let  ir(t,j)  be  the  appointment 
x  to  y^  we  hire  only  in  rank 


ir(t,J)  at  time  t  .  If  |x^  (t)  ,u^  <t)| l**dt  from  x  to  y^  at  cost  WA 

then  |x(t)  »u(t)|^Q  leads  from  x  to  y  at  cost  £  X^W^  ,  where 

u(t)  -  l  X“uj(t) 

J  2 


x(t)  -  l  X®x^(t)  . 
J  2 


The  justification  of  our  algorithm  is  as  follows.  Let  y  be  the  optimal 

m  m  i 

solution  of  (2).  For  any  m  ,  if  X  is  optimal  for  (3),  then  £  A  y  is 

i-1  1 

feasible  for  (2).  It  is  easy  to  show  that  W(y)  is  convex,  hence 
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w(y)  <  w(  l  xV)  <  l  x“w  . 

vi»l  /  i-1 

In  addition  the  duality  Theorem  of  linear  programing  implies  rm  >  0  and, 

W(y)  <  om  -  [  XV  . 

i-1  1 

Thus  o"  is  an  upper  bound  on  the  optimal  value  of  (2).  Since  rm  >  0  ,  and 
yA  >  0  we  have  yAr°  >  0  .  In  addition  y  is  feasible  for  (4)  so 

v"*1  -  Min  W(y)  -  yArB  <  W(y)  -  yAr°  <  W(y)  . 

Therefore  v  is  a  lower  bound  on  the  optimal  value  U(y)  of  (2) . 

o+l  n 

The  stopping  condition,  v  >  o  ,  implies  our  current  lower  bound  equals 

the  upper  bound,  and  we  have  found  the  optimal  solution. 

The  proof  is  completed  by  shoving  problem  (4)  can  be  solved  by  setting 

q  -  Arm  and  using  the  dynamic  programming  procedure  developed  in  Section  I. 

Let  v  and  y  be  the  value  and  terminal  point  from  the  dynamic  programming 

is  mi  j. 

calculation.  Suppose  there  exists  a  y  such  that  W(y)  -  yAr  <  v  .  Then  the 

T-l 

program  x(t),u(t)  ■  g  leading  to  y  would  be  optimal  for  the  dynamic 

D^l 

program.  This  contradicts  the  definition  of  v  .It  is  also  apparent  that 

v"*1  -  WCy”^1)  -  y’^Ar0  .  If  there  is  a  cheaper  way  to  y®^  we  could  do 
b4*1 

better  than  v  in  the  dynamic  program.  Note  the  dynamic  program  always 

T 

generates  an  extreme  point  of  R  (x)  ,  since  It  appoints  in  only  one  rank  in 
T 

each  period.  R  (x)  has  a  finite  number  of  extreme  pointe,  therefore  we  must 

converge  to  an  optimal  solution  in  a  finite  number  of  steps.  This  completes 

our  discussion  of  the  generalized  linear  program. 

The  feasibility  of  (1)  is  checked  in  a  similar  way.  First  set  q  -  0  , 

1  T 

then  find  a  y  c  R  (x)  and  -  W(y^)  -  Min  W(y)  .  For  any  m  solve  a 


phase  I  master. 


K 

(5)  Minimize  '  z, 

k-1  k 

D 

Subject  to  £  X  (y .A)  +  z  >  0 
i-1  11 

X  >  0  ,  x  >  0  . 

■  m 

K  ■ 

Let  (X*za)  solve  (5).  If  [  r  ■  0  then  £  X  y  is  feasible  for  (2). 

k-1  *  i-1  1  1 

Othend.se  let  (rBfoa)  be  the  optimal  dual  solution  of  (5)  and  solve  the 
dynamic  program  In  Section  1  with  c(t)  -  d(t)  -  0  for  all  t  and  q  -  Ara  . 

Let  y  be  the  terminal  point  of  the  dynamic  program.  If  o  >  -  j  Ar 

B  wai.1  m 

than  add  column  (0,y  A)  to  problem  (5).  If  <J  <  -  y  Ar  then  stop: 
problem  (2)  ie  infeaeibUj. 

The  Justification  is  similar  to  the  phase  II  problem  described  previously. 
When  switching  from  phase  I  to  phase  II  ve  need  to  know  W^  for  1  -  1,2,  ...,  a 
When  we  solve  the  dynamic  programming  subproblem  In  phase  I  we  should  calculate 
the  cost  of  going  from  x  to  y*  using  the  optimal  program.  This  is  an  upper 
bound  (usually  exact)  on  .  A  careful  review  of  the  phase  II  procedure  should 
convince  the  reader  that  it  la  possible  to  substitute  these  bounds  for  the  W^ 
and  comnmnce  phase  II  directly.  If  one  of  the  bounds  is  not  tight,  then  the 
column  will  be  regenerated  If  It  is  needed  in  the  course  of  calculations. 

Problem  (1)  can  be  considered  as  a  linear  program  with  T  +  n  equality  con¬ 
straints  and  n  *  (T  +  1)  nonnegative  variables.  In  contrast  problem  (3)  has 
n  +  1  equality  constraints  and  n  +  M  variables,  where  M  Is  the  number  of 
extreme  points  of  R  (x)  .  When  T  -  30  ,  and  n  -  15  ,  then  (1)  has  45  con¬ 


straints  to  16  constraints  in  (3). 
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We  elected  to  solve  problem  (3)  for  several  reasons 
the  dynamic  programming  subroutine,  the  reduced  storage, 
arrays,  and  the  flexibility  in  changing  the  data. 


the  efficiency  of 
the  ease  of  handling 
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III.  INFINITE  HORIZON  PROBLEMS 

This  section  considers  stationary  problems  with  an  infinite  planning  horizon 
and  with  constraints  on  the  staff  distributions  in  each  period.  In  practice  most 
planning  problems  do  not  have  a  specified  finite  horizon;  many  manpower  policy 
decisions  have  such  long  term  effects  that  decisionmakers  must  consider  the  long 
run  implications  of  any  policy.  One  way  to  examine  the  long  run  effects  of  present 
decisions  is  to  select  a  large  planning  horizon  T  with  the  hope  that  the  impact 
of  present  decisions  on  periods  t  >  T  is  likely  to  be  small.  This  approach  has 
two  drawbacks:  it  is  difficult  to  select  an  appropriate  T  ,  and  if  T  is  large 
the  resulting  multi-stage  planning  problem  is  difficult  to  solve.  Our  alternative 
approach  is  to  assume  an  infinite  planning  horizon  in  a  stationary  environment. 
Thus,  with  the  exception  of  the  state  vector  x  ,  every  period  we  begin  a  new 
problem  that  is  identical  to  the  problem  we  faced  at  time  zero.  These  stationary 
problems  are  frequently  easier  to  solve  than  long  finite  horizon  problems. 

In  each  time  period  the  system  will  obey  the  law  of  motion,  the  no  firing 
condition,  and  the  growth  constraint.  In  addition  we  shall  require  that  for 
each  t  the  staff  distribution  x(t)  is  contained  in  the  polyhedral  cone 
C  *«  {y  |  yA  j>  0}  . 

Theorem  2  describes  theoretical  conditions  necessary  and  sufficient  for  a 
sequence  of  staff  distributions  to  remain  in  C  indefinitely.  From  these  con¬ 
ditions  we  develop  in  Theorem  3  a  test  for  the  initial  staff  distribution  which 
will  guarantee  the  existence  of  feasible  solutions  to  the  infinite  horizon  mini¬ 
mum  cost  problem.  Two  objective  criteria  are  considered — long  run  average  cost 
and  total  discounted  cost,  and  for  both  cases  bounds  on  the  minimum  cost  have 
been  calculated.  These  results,  which  are  contained  in  Theorems  4  and  5  respec¬ 
tively,  were  accomplished  with  the  aid  of  a  surrogate  linear  program.  The  sur¬ 
rogate  program  defines  a  stationary  linear  appointment  policy  u(t)  ■  x(t)D 


\ 


t 


x(t) 

case  the  lower  bound  is  not  exact.  In  any  case  we  have  y*  e  C 

x(t)f 


which  obeys  the  law  of  motion,  no  firing,  and  growth  constraints.  Given  an 
initial  distribution  x(0)  ,  the  appointment  policy  u  =  xD  determines  the 
entire  sequence  x(t)  .  If  x(t)  e  C  for  all  t  then  the  stationary  linear 
appointment  policy  is  optimal  and  the  surrogate  linear  program's  lower  bound 
is  exact.  It  is,  however,  possible  that  x(t)  i  C  for  some  t  .  In  this 

as 

t  -*•  0  ,  and  in  addition  a  weighted  average  (which  depends  on  the  discount  fac¬ 
tor)  of  the  x(t)  is  in  C  .  These  properties  lead  us  to  suspect  that,  even 
if  x(t)  t  C  for  some  t  ,  our  lower  bound  is  accurate  and  our  easily  imple¬ 
mented  appointment  policy  is  nearly  optimal. 

As  we  point  out  at  the  close  of  this  section  it  is  theoretically  possible 
to  find  the  actual  optimal  policy.  However  it  is  a  difficult  task  and  not  a 
task  we  would  like  to  repeat  to  gain  sensitivity  information.  What  is  more,  it 
will  not  in  general  lead  to  an  easily  implemented  linear  decision  rule. 

The  infinite  horizon  constraints  are: 


x(t)  -  x(t  -  1)Q  +  u(t  -  1) 

(1)  x(t) f  -  «Cxf 

u(t  -  1)  >  0 

x(t)A  >0  t  -  0,1,2,  ... 

| 

x(0)  -  x  j>  0  given. 

The  use  of  Q  and  6  above  Instead  of  P  and  8  as  before  is  in  anticipation 
of  Section  IV,  where  different  policies  will  be  considered  for  promotion  and 
growth  in  the  transient  and  stationary  stages  respectively. 

The  first  three  constraints  enforce  the  law  of  motion,  the  constant  growth 
rate  of  x(t)f  ,  and  the  no  firing  condition.  The  final  constraint  restricts 

* 

1 


the  distribution  x(t)  to  the  cone  C  -  {y  |  yA  >  0}  for  all  t  . 
The  constraints  can  be  written  in  terms  of  x(t)  alone. 


x(t)  >  x(t  -  1)Q 

(2)  x(t)f  -  fi^cf 

x(t)A  >0  t  -  1,2,  ... 

x(0)  ■  ^  0  given 

We  can  also  write  the  constraints  in  terms  of  u(t)  ,  since 

x(t  +  1)  -  xQt+1  +  l  u(s)Qt”8  . 

s-0 

We  have 

(3)  l  u(s)Qt_Sf  -  x[fit+1I  -  Qt+1]f 

s-0 

l  u(s)Qt“SA  >  -  xQt+1A 

s-0 

u(t)  ^  0  t  ■  0|1|2)  ••• 

m 

The  objective  is  to  minimize  £  at[x(t)c  +  u(t)d]  .  When  a* S  <1  , 

t-0 

this  sum  is  finite.  If  a* 6  -  1  ,  the  cost  is  typically  infinite.  In  this 
case  we  seek  to  minimize 

T-l 

lim  sup  y  l  ot(x(t)c  +  u(t)d]  . 

T-*»  t-0 


(4) 
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Before  searching  for  optimal  policies  we  must  examine  the  question  of 

feasibility.  Given  x  e  C  is  it  possible  to  remain  in  C  indefinitely? 

This  question  cannot  be  answered  precisely.  However,  we  do  obtain  a  sufficient 

00 

condition  for  x  e  C  in  Theorem  3,  and  a  necessary  condition  from  Theorem  5. 
Both  conditions  require  the  solution  of  a  linear  program. 

Recall  from  the  introduction  that  E  -  {y  |  6y  _>  yQ,y  _>  0}  .  The  following 
lemma  is  extremely  useful  in  analyzing  questions  of  feasibility. 

Lemma  1: 

If  {x(t)Jt_Q  *9  feasible  for  (2),  define. 

T-l 

y  -  lim  sup  ^  I  $  ^(t) 

T-h*>  1  t-0 

then  y  e  CHSflE  . 

Proof : 

For  each  T  let 


l  1-1  «. 

y(T)  -  £  l  5~x(t)  . 
t-0 

y(T)  is  a  nonnegative  combination  of  points  in  C  ,  therefore  y(T)  is  in 
C  .  For  each  t  ,  6  fcx(t)  e  S  -  {z  |  zf  -  xf,z  >  0}  ,  thus  y(T)  e  S  ,  since 
~?(T)  is  a  convex  combination  of  points  in  S  .  It  is  apparent  that  y  t  COS  . 

From  the  constraints  of  (2),  6  Cx(t)  >  [6  ^  *^x(t  -  1)Q]  .  Summing  the 

above  relation  for  t-0,  T  -  1  (with  x(-l)  -  0)  ,  we  obtain 

y<T>  >  |f  y(T)Q  -  6  (T  1)^T  ~  1?<*j . 

-(T-l)  1 

Now  6  x(T  -  1)Q  remains  bounded,  therefore  as  T  -*•  »  ,  y  >  yQ  .  Also, 
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since  6  Cx(t)  <_  xf(min  f.)  for  each  t  ,  we  have  y  <  «  .  Thus  y  e  E  .  |  | 

i 

Recall  R(x)  ■  {y  |  y  >,  xQ,yf  ■  <5xf}  .  Let  C°  =  C  ,  and  for  t  =  1,2,  ..., 
define 


CC  ■  {x  |  x  e  C  and  R(x)  H  CC  ^  +  $}  . 

C*'  is  the  set  of  starting  points  x  such  that  it  is  possible  to  remain  in  C 

t  t+1  mm  t  oo 

for  t  steps.  Clearly  C  D  C  .  Let  C  ■  Qj  c  •  Then  C  is  the  set 
of  feasible  starting  points  for  problem  (1).  It  follows  that 


Lemma  2: 

x  e  C  if  and  only  if  x  e  C  and  R(x)  DC  j4  <j>  . 

Theorem  2  below  gives  an  operational  check  on  whether  feasible  initial  states 

exist.  It  does  not  answer  the  more  difficult  question  of  whether  a  particular 

00 

starting  state  x  is  in  C  . 

Theorem  2; 

c"  is  void  if  and  only  if  COE  is  void. 


Proof: 


If  x  c  C  HE  ,  then  x  e  C  since  6Cx  is  feasible  for  (2). 

00 

For  the  converse,  suppose  .  x  e  C  .  From  Lemma  2  we  can  construct  a 

sequence  |x(t)jt-0  such  that  x(0)  ■  x  and  x(t  +  1)  z  R[x(t)]  OC*  for 

t  ■  0,1 .  From  Lemma  1  we  can  then  find  a  y  such  that  y  e  COE  .  1 1 

As  mentioned  above  these  results  do  not  give  an  explicit  characterization 
00 

of  C  ,  the  set  of  feasible  starting  points.  In  some  cases  it  is  possible  to 
prove  that  C-  ■  C  .  Let  {t^lk-l  *>e  t*1*  ^treme  rays  of  the  cone  C  .  Then 


C  - 


|y  I  y  >  l 

(  k-1 


w 
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If  R(*k)  f'1  C  j*  4>  for  all  k  ,  then  R(y)  0  0^4)  for  each  y  e  C  .  Therefore 

=  C  ,  a  condition  which  leads  to  the  identity  C  =  C  .  In  Theorem  3  we  apply 

this  result  to  develop  a  means  of  testing  whether  a  given  value  of  x  e  C  is  a 
00 

member  of  C  . 

Theorem  3: 

00 

If  x  c  C  ,  then  x  c  C  if  the  linear  program  (5) ,  below,  has  an  optimal 
* 

solution  A  <  1  . 

(5)  Minimize  A 

Subject  to  Ax  +  u  -  y  ■»  0 

yr  -  6xf 
y  >  xQ 
uA  _>  0 
u(6I  -  Q)  _>  0 
^iO,u>0,y>0. 

Proof : 

The  constraints  of  (5)  require  that  u  e  C  D  E  and  that  y  e  R(x)  .  Let 

*  *  *  a  „  it  it 

(A  ,u  ,y  )  solve  (5),  with  A  <  1  .  Then  define  u  to  satisfy  (1  -  A  ){J  ■  u  . 

*  *  * 

Obviously  0  t  C  HE  and  y  ■  A  x  +  (1  -  A  )5  ,  Now  consider  a  new  problem 

with  C  -  {y  |  y  -  UjX  +  i^u,^  1  O.V2  -  *  since  x  and  5_1u  are  in  c  * 

we  have  C  C  C  ,  and  C*C  C*  .  However  y*  e  Cfl  R(x)  and  6u  e  C  nR(u')  , 

therefore  C*  •  C1  ■  C  ,  and  x  e  C*CC*  .  1 1 

Note  that  problem  (5)  contains  3  *  N  +  K  +  1  constraints  with  2N  +  1 


Z3 


nonnegative  variables.  The  dual  problem  of  (5)  will  have  only  2N  +  1  constraints 

t 

! 

and  thus  will  be  easier  to  solve. 


Nov  we  turn  to  optimization  and  consider  the  average  cost  problem  (ad  -  1)  . 

! 

f 

If  we  substitute  u(t)  ■  x(t  +  1)  -  x(t)Q  in  (4)  ve  obtain 


1  r  t 

Minimize  11m  aup  =  £  a  x(t)g  . 

T-*»  1  t«0 

Subject  to  x(t)  >  x(t  -  1)Q  , 
x(t)f  -  6Cxf  , 

x(t)A  >0  ,  t  -  0,1,2,  ... 

x(0)  -  x  £  0  given 
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where  g  »  c  +  6d  -  Qd  .  Since  6  a  a*"  ,  we  know  from  Lemma  1  that  the 
value  of  any  solution  jx(t)jt.Q  *s  giv®o  by  yg  where 

1  T_1  -t 

y  -  lim  sup  ±  l  6  x(t)  , 

T-h.  1  t-0 

and  yeCDSOE.  We  can  therefore  calculate  a  lower  bound  on  the  value  of  (6) 
by  solving: 

(7)  Minimize  yg 

Subject  to  y  >  y  Q 

yf  ■  xf 

yA  >  0 
y  >  0 

Problem  (7)  can  be  written  In  a  more  operational  form.  Since,  y  -  yaQ  +  u  , 
we  set  y  ■  u(I  -  aQ)  ^  -  uB  where  B  >  0  .  Problem  (7)  becomes; 

(8)  Minimise  uBg 

Subject  to  uBf  ■  xf 

uBA  >  0 

u  >  0  . 

Theorem  4: 

If  v  -  [61  -  Q ] f  >  0  end  06  ■  1  ,  then 

(1)  The  optimal  value  of  (8)  is  a  lower  bound  on  the  optimal  value  of  (6). 
(11)  If  we  maximise  In  (8)  we  obtain  an  upper  bound  on  the  optimal  value 
of  (6). 


(Hi)  If  x  is  in  the  relative  interior  of  E  fl  C  ,  then  the  bound  obtained 
in  (8)  is  tight  and  an  optimal  policy  can  be  described. 

(iv)  If  u*  is  the  optimal  solution  of  (8)  then  the  stationary  linear 
appointment  policy  is 

* 

u(t)  -  x(t)  -  x(t)D 

u  f 

Proof : 

Items  (i)  and  (ii)  follow  from  the  discussion  above  and  Lemma  2. 

We  shall  sketch  a  proof  of  item  (ill).  Let  u  solve  (8)  ,  y  -  u  B  , 

* 

and  let  L  be  the  line  segment  joining  x  and  y  e  S  .  For  each  t  let 

z(t)  “  afcx(t)  e  S  .  Given  z(t)  the  optimal  policy  is  to  choose  z(t  +  1) 

as  the  point  in  L  fl  R[oz(t)]  that  is  closest  to  y  .  This  rule  generates 

a  sequence  of  points  z(t)  e  C  that  converges  to  y  . 

Item  (lv)  warrants  more  discussion  than  proof.  First,  note  that  v  is 
*  * 

a  column  vector  and  u  is  a  row  vector,  therefore  vu  is  an  n  x  n  matrix 
and  u*f  is  a  scalar.  We  can  easily  verify  that  this  appointment  policy  leads 
to  a  sequence  x(t)  that  satisfies 

x(t)  >  x (t  -  1)Q 

and 

x(t)f  -  «Cxf 

for  all  t  .  In  addition  let  z(t)  ■  aCx(t)  .  Since  x(t  +  1)  ■  x(t)[Q  +  D] 
we  have  z(t  +  1)  ■  z(t)[a(Q  +  D)]  ■  z(t)G  where  z(t)  t  S  for  all  t  .  It 
is  easy  to  see  that  z(t)  “  xG  . 

* 

G  is  a  square  nonnegative  matrix.  The  vectors  f  and  y  are  easily 
seen  to  be  right  and  left  eigenvectors  of  G  corresponding  to  an  eigenvalue 
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of  1  .  Let  F  be  a  diagonal  matrix  with  elements  on  the  diagonal. 

Then  F  'GF  is  a  Markov  matrix.  We  shall  assume  that  this  Markov  matrix 

corresponds  to  a  chain  with  a  single  ergodic  class  ([5],  page  69).  This  con- 

* 

dition  is  satisfied  if,  for  example,  >  0  and  it  is  possible  to  reach 

any  rank  starting  at  rank  1.  Let  it  be  the  unique  probability  vector  ([5], 

-1  * 
page  71),  that  solves  it  -  irF  GF  .  One  can  check  that  ■  f^y^  . 

In  addition  (F  ^GF) ■*  H  where  each  row  of  H  is  equal  to  ir  .  Then 

Gfc  -  FfF^GFjV1  -*■  FHF-1  .  Thus  z(t)  -  xGC  xFHF_1  -  y*  . 

This  establishes  two  facts.  The  stationary  linear  policy  achieves  the 

lower  bound  determined  by  (8)  and  leads  to  a  sequence  of  states  x(t)  such 

that 

It  is  a  conjecture  that  the  optimal  value  function  of  problem  (6)  is 

09 

constant  for  initial  states  x  in  the  relative  interior  of  C  .  It  is 

09 

possible  to  get  stuck  on  the  boundary  of  C  and  thus  incur  a  higher  per 
period  cost.  Note  that  If  the  conjecture  above  is  true  it  is  not  operational 

09 

since  it  requires  a  precise  characterization  of  C  and  the  specification  of 
an  optimal  appointment  policy. 

We  now  turn  our  attention  to  the  problem  of  minimizing  discounted  cost. 

In  the  discounted  case  we  shall  frequently  use  the  identity 

99  99 

l  aCx(t)  ■  xB  +  a  £  atu(t)B 
t-0  t-0 

where  B  -  (I  -  aQ)  *  .  Using  this  identity  with  constraints  (3)  and  the  dis¬ 
counted  cost  objective  we  obtain 


(13) 


Minimize 


xBc  +  £  c^uCtHaBc  +  d] 

t=0 


Subject  to  £  u(s)Q  8  f  ■  x[6  I  -  QC]f 
8*0 


l  u^xf^A  >  -  xQS 

8*0 


u (t )  >0  t  »  0,1,2,  ... 


Define  u  *  £  atu(t)  .  Then  £  afcx(t)  *  xB  +  auB  ,  and  the  objective 

t-0  t=0 

value  of  any  solution  in  (9)  is 


xBc  +  u[aBc  +  d]  . 


We  wist  to  substitute  a  finite  set  of  constraints  for  the  infinite  set  (9). 
Instead  of  writing 


t-i  . 

I  u(s)Qt_S'1A  >  -  xQCA 
x-0 


for  all  t  *  0,  ...  ,  we  multiply  the  tth  constraint  by  a1  and  sum.  The 


resulting  problem  is 

(10) 


Minimize  u[aBc  +  d]  +  xBc 


Subject  to  u[aBf]  ■  - - 7  -  xBf 

1  —  ao 


u[aBA]  >  -  xBA 

u  >  0 


This  problem  has  K  +  1  constraints,  within  nonnegative  variables.  We  know 
that  any  feasible  solution  of  (9)  determines  a  feasible  solution  of  (10). 


Theorem  5: 

If  v  -  [61  —  Q]f  >  0  ,  and  06  <  1  ,  then 

(1)  Infeasibility  of  (10)  Implies  x  i  C-  . 

(11)  If  u*  solves  (10)  y  then  u*[aBc  +  d]  +  xBc  is  a  lover  bound  on  the 
optimal  value  of  (9)  . 

(Hi)  The  stationary  linear  appointment  policy  Is 

u(t)  -  x(t)  -  x(t)D  . 


Proof : 


Parte  (1)  and  (11)  follow  since  any 


u  «  J  atu(t)  is  feasible  in  (10). 
t-0 


feasible  In  (9)  implies 


In  part  (Hi)  it  is  easy  to  verify  that  the  stationary  linear  decision  rule 

leads  to  a  sequence  of  distributions  x(t)  which  satisfy  the  law  of  motion 

and  growth  constraint.  As  before  we  can  also  show  that  in  the  limit 

„  x(t)f 

is  in  C  .  In  addition  £  a*x(t)A  >  0  .  Let  u(t)  be  the  appointment 

t-0 

policy  resulting  from  the  stationary  linear  appointment  rule.  Note  that 


u  -  l  atu(t) 
t-0 


1 

* 

u  f 


m 

l  etx(t)v 
t-0 


* 


u 


* 

nu 


where  n  is  a  constant.  Recall  that  the  sequence  u(t)  is  designed  so 
that  u(t)f  •  6fcxf  for  all  t  .  Therefore  u  is  feasible  for  (10)  and 


£  ft  if 

u[aBf]  ■  nu  [aBf]  ■  1  .  However,  u  ia  feasible  for  (9),  thus  u  (aBf]  »  1  , 
so  n  ■  1  and  u  ■  u*  .  Thus  Che  scationary  linear  policy  achieve#  the  lower 
bound  calculated  by  (10).  || 

* 

We  say  a  distribution  of  staff  y  is  stationary  if  the  stationary  linear 
decision  rule  u(t)  -  x(t)D  applied  at  x(0)  ■  y  leads  to  the  sequence 
x(t)  -  «V  • 

h 

In  the  average  cost  case  the  stationary  staff  distribution  y  is  given 
*  *  * 

by  y  -  u  B  ,  and  the  optimal  cost  by  y  g  .  In  the  average  cost  case  to  find 
the  stationary  staff  distribution  we  must  solve. 


* 

y  f  -  xf 

This  system  of  equations  is  similar  to  the  characterization  of  the  limiting 
distribution  in  a  finite,  regular,  Markov  chain  [5].  The  system  can  be  solved 
in  the  same  manner. 

As  a  final  comment  let  W(x)  be  the  optimal  value  of  problem  (9)  and 
V(x)  the  optimal  value  of  problem  (10)  .  We  know  V(x)  <  W(x)  ,  and  W(x) 
satisfies  a  functional  equation 

W(x)  -  Min  fxc  +  du  +  aW(xQ  +  u]  I  , 

u>0  L  J 

uf«xv 

uA>-xQA 

where  W(x)  -  +■  if  x  i  C  It  is  theoretically  possible  using  a  generalization 
of  Howard's  policy  iteration  scheme  (see  {9J)  to  compute  V(x)  for  x  c  S  and  u(x) 
an  optimal  policy.  We  suspect  that  W(x)  and  u(x)  are  piecewise  linear  for 

m 

x  c  C  . 


IV.  TRANSIENT-STATIONARY  PROBLEMS 


New  institutions  or  institutions  experiencing  severe  change  often  undergo  a 
transient  period  before  reaching  a  state  of  equilibrium  growth.  This  section 
shows  how  the  infinite  horizon  model  of  Section  III  can  be  combined  with  results 
from  Sections  I  and  II  to  calculate  lower  bounds  on  the  minimum  cost  of  the 
combined  transient  and  stationary  problem. 

In  a  typical  transient  problem  the  system  grows  rapidly  during  the  first 
T  years  at  10Z  per  year.  After  reaching  maturity  the  system  may  grow  at  a 
reduced  rate  of  1Z  per  year.  In  this  section  we  provide  a  method  for  analyzing 
transient  problems  which  will  allow  a  decision  maker  to  gauge  the  tradeoffs 
between  the  initial  growth  rate  and  the  duration  of  the  transient  period. 

Let  P  be  the  promotion  matrix  during  the  transition  years  and  Q  the 
promotion  matrix  during  the  steady  state  period.  P  and  Q  may  not  be  equal 
because  of  different  promotion  policies  in  the  two  periods.  Let  6  and  6 
be  the  growth  rates  during  the  different  periods. 

Now  consider  a  problem  with  a  T  year  transient  period,  and  assume  no 
restriction  is  placed  on  the  distribution  x(t)  during  the  transient  stage. 

In  the  steady  state  from  period  T  onward  we  impose  the  restriction 
x(t)  e  C  *  {y  |  yA  >  0}  for  t  >  T  . 

As  in  Section  II,  let  W[y]  be  the  minimum  cost  of  going  from  x  to  y 
in  T  periods.  Let  V[y]  be  the  optimal  value  of  problem  (10)  in  Section  III. 
V[y]  is  a  lower  bound  on  the  minimum  cost  operating  the  system  over  an  infinite 
horizon  with  initial  state  y  . 

V(y)  ■  yBc  +  Min  u[aBc  +  d] 

Subject  to  u[aBf  J  ■  j  “!? a&  ~  Y®* 


u[aBA]  >  -yBA 
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u  >  0 


To  calculate  a  lower  bound  on  the  transient  and  stationary  phase  of  operation 
we  must  solve 

(2)  Minimize  W(y)  +  aTV(y) 

T 

Subject  to  y  e  R  (x)f"IC  . 


We  outline  a  procedure  for  solving  (2)  below.  The  method  follows  the 
generalized  programming  technique  of  Section  II.  It  is  important  to  find  a 
technique  that  will  produce  an  optimal  or  nearly  optimal  solution  of  (2) 
efficiently.  The  efficiency  in  solution  allows  one  to  solve  (2)  repeatedly 
and  to  carry  out  sensitivity  analyses. 

The  master  program  is: 

5  T  T+l  T 

(3)  Minimize  I  X  (W  +  y.o  Be)  +  ulo  Be  +  a  d] 

i-1  1  1  1 


Subject  to: 


m 

l 

i-1  X  X 


>  0 


•m 

i-1  X 


-  1 


l  X  y  BA  +  uaBA  >  0 
i-1  1 


I  xi(Ti“>  +  ■  p 


X.  >  0  u  >  0 

i  -  - 

where  the  y^  are  points  in  R^(x)  and  -  WCy^)  . 

Let  (Xm,  «■)  solve  (3)  and  let  (rB,oB,sB,tB)  be  the  optimal  dual  solution. 


We  have: 


(4)  (i)  (aBA)sm  +  (aBf)i(»m  <  (a^)(aBc  +  d) 


(il)  (y^A)^  +  om  +  (yjBAjs®  +  (y^Bf)^®  <  (W^  +  y^Bc) 

for  i  -  1,2 . ■ 

(111)  r®  >  0,  s®  >  0 

m 

(iv)  a®  +  pi|»®  -  l  A®(W.  +  y.aTBc)  +  u“[(aT)(aBc  +  d))  . 

1-1  1  1 

The  first  three  conditions  state  that  (r”,o®,s®,i|»®)  Is  dual  feasible. 

The  final  condition  is  that  both  solutions  are  optimal.. 

m  a 

As  In  Section  II,  a  +  p  provides  an  upper  bound  on  the  optimal  value 

■ 

of  (2).  The  optimality  of  our  current  solution,  J  A®y  is  checked  using  the 

1-1  1  1 

■ubproblem: 

(5)  Minimise  W(y)  -  y(Ar*  -f  BAs*  +  Bf<>*  -  aTBc] 

Subject  to  ye  RT(x) 

The  optimal  solution  of  (S)  Is  calculated  using  the  dynamic  program  with 

q*  -  Ar*  +  BAs-  +  Bf^n  -  aTBc  .  Lee  v***  be  the  optimal  value  of  (5)  and 
qf4*1 

y  the  terminal  point. 

The  value  v  can  be  used  to  calculate  a  lower  bound  on  the  optimal 
value  of  (2).  Let  V*  be  the  optimal  value  of  (2),  then 

o*  +  p*p  >  V*  >  v*41  +  p*p 

There  are  two  cases  to  consider: 

1.  If  s>  >  o  ,  then  the  current  solution  y  -  £  A®y.  solves  (2). 

1-1  1  1 

2.  If  nP*1  <  ,  then  the  column  (tf^  +  aTy*f XBc ,  y^A,  1,  ymflBA,  y®flBf) 

can  enter  the  basis  In  (3). 
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To  verify  the  lower  bound  claim  consider  an  alternate  way  of  expressing 
T 

a  V(y)  using  the  duality  theorem. 

oTV(y)  ■  oTyBc  +  Max  i>(p  -  yBf)  -  yBAs 

(5)  Subject  to  (aBf)tj;  +  (aBA)s  <  aT(aBc  +  d) 

s  >  0 

m 

According  to  4(1)  i (i®  and  sm  are  feasible  for  (5) ,  thus: 

(6)  aVy)  >  oTyBc  +  </®(p  -  yBf)  -  yBAs® 

for  all  y  .  In  particular  for  y  ,  the  optimal  solution  of  (2). 

Since  the  optimal  solution  y  c  C  ,  and  r®  >  0  ,  we  have  yAr®  >  0  .  Thus 

(7)  <y)  >  V(?)  ~  yAr®  . 

Combining  (6),  with  y  m  y,  and  (7)  ve  obtain 

V*  -  W(y)  +  aTV(y)  >  W(y)  -  y[Ar®  +  BAs®  +  Bf*“  -  aTBc]  +  *“p 
W(y')  -  y"[q“)  +  4>*p  -  v®fl  +  i>mp 

We  can  use  the  technique  described  in  Section  II,  to  obtain  »  first  feasible 
solution  to  problem  (3) . 

It  is  possible  to  reformulate  (2)  as  a  linear  program  with  T  +  2  *  n  con¬ 
straints  and  n  *  (T  +  2)  variables.  For  reasons  identical  to  thoss  given  in 
Section  II  and  the  introduction,  ve  preferred  the  generalized  linear  programming 
approach. 


V.  USE  OF  THE  MODEL 


Our  model  was  originally  motivated  by  an  examination  of  a  university 
faculty  system.  This  section  demonstrates  how  the  model  can  be  applied  to 
a  faculty  system  with  three  ranks  of  members:  assistant  professors,  associate 
professors,  and  full  professors.  This  highly  aggregated  example  was  selected 
to  facilitate  geometrical  display  of  the  results  and  more  importantly,  to  stress 
our  contention  that  thie  type  of  model  is  best  suited  to  evaluate  and  suggest 
general  policies  or  to  distinguish  the  effects  of  alternate  policies.  We  do 
not  view  our  model  as  a  precise  control  instrument.  In  the  university  context: 
we  are  not  trying  to  specify  appointment  quotas  in  each  department  for  each 
year,  rather  we  are  concerned  with  judging  the  long  range  financial  and  staffing 
effects  of  alternate  promotion,  eelary,  and  retirement  policies. 

A  small  model  haa  the  advantage  of  flexibility  in  altering  the  structural 
coefficients.  For  example,  the  promotion  rates,  P^  ,  are  actually  the  result 
of  policy  decisions.  If  a  change  in  the  promotion  policy  is  being  considered, 
it  la  relatively  easy  to  alter  the  and,  resolve  the  problem,  and  judge 

the  impact  of  the  new  promotion  policy.  A  more  disaggregated  model  might 
represent  the  dynamics  of  the  syatam  more  accurately,  but  we  rapidly  lose 
flexibility  as  the  number  of  renks  increases. 

The  elgorlthms  were  programmed  in  APL  360  %'itb  the  aid  of  Edvard  Stohr 
and  then  used  to  solve  two  classes  of  problems. 

To  begin,  we  examined  the  problem  of  reaching  a  specified  distribution  of 
faculty  in  15  years  at  minimum  cost.  This  problem  was  solved  under  four  separate 
assumptions  concerning  promotion  and  retention  policy  and  salary  structure.  In 
each  of  the  four  cases  we  varied  the  target  distribution  over  a  wide  range  of 
possibilities.  Our  computational  efforts  involved  about  150  of  the  generalized 
linear  programs  described  in  Section  II.  Since  each  generalized  linear  program 
used  the  dynamic  programming  subroutine  about  14  times, this  calculation  included 


solving  200P  of  the  dynamic  programs  described  in  Section  1.  Ue  shall  present 

one  set  of  results  in  detail  and  comment  on  others. 

* 

The  following  data  were  used. 

Initial  distribution:  •  .3  ,  x2  ■  .3  ,  ■  .4 

Discount  factor  1:  i.e.  total  cost. 

Growth  rate  0  (constant  size). 

Support  cost  (20,28,34) 

Recruiting  and  hiring  cost  (2,2,2) 

Time  horizon:  15  years. 

Promotion  Matrix 


In  Figure  1  below,  y^  and  y2  refer  to  the  target  values  of  assistant 
and  associate  professors.  Obviously,  the  fraction  of  full  professors  is  given 
by  yj  *  1  -  yj  -  y2  .  The  dark  border  shows  the  boundary  of  distributions  that 
can  be  obtained  from  x  in  15  periods  (R^fr)).  The  dotted  straight  lines  indi¬ 
cate  the  boundary  of  the  equilibrium  set.  If  a  distribution  y  is  in  the  equil¬ 
ibrium  set,  then  there  exists  a  hiring  policy  such  that  the  distribution  y  can 
be  maintained  from  year  to  year.  The  curved  lines  in  Figure  1  are  Isocost  lines: 
any  two  targets  on  the  same  isocost  line  can  be  reached  for  the  same  minimum  cost. 
Note  the  Isocost  lines  as  drawn  do  not  differ  by  a  constant  amount.  The  cost 
function  is  actually  bowl  shaped.  It  is  fairly  flat  over  a  large  region,  and 
has  relatively  steep  sides.  In  particular,  as  y3  Increases,  the  cost  increases 
sharply.  Note  also  that  the  cost  Increases  when  the  target  point  is  outside  of 
the  equilibrium  set.  This  is  due  to  the  difficulty  the  system  has  in  reaching 

a  nonequilibrium  target. 

- 

One  should  not  attach  undue  importance  to  these  figures;  our  purpose  is 
to  give  an  example  of  the  models'  usefulness  and  not  to  solve  a  particular 
problem  at  a  particular  university. 


The  number*  on  Che  isocost  lines  are  the  cost  per  faculty  member  in 
thousands  of  dollars  of  running  the  system  for  IS  years.  To  estimate  the 
total  cost  you  must  multiply  by  the  number  of  faculty  members. 

The  costs  of  reaching  various  target  points  were  calculated  under  three 
alternative  set  promotion  and  salary  policies.  First  we  changed  the  first  top 
row  of  the  promotion  matrix  to  .666  .1666  .0  .  This  change  had  two  effects: 
it  Increased  costs  by  about  3Z,  and  it  raduced  the  size  of  the  feasible  region. 
Targets  with  >  .34  became  infeasible.  Thus  a  more  liberal  promotion  policy 
did  not  cause  a  drastic  Increase  in  costs,  however  it  did  cause  the  loss  of  a 
great  amount  of  flaxlblllty.  In  the  next  trial  we  moved  to  a  more  stringent 
promotion  policy  and  changad  tha  top  row  of  tha  promotion  matrix  to  .65  .08  .0 

This  resulted  in  a  2X  decreaaa  in  costs  and  an  enlargement  of  the  set  of  feasible 
target.  The  final  application  retained  the  more  stringent  promotion  policy  and 
raised  the  support  cost  of  full  professors  to  40  .  This,  of  course,  did  not 
change  the  set  of  feasible  targets,  but  it  did  cauaa  an  8Z  Increase  in  cost. 

The  second  major  application  used  the  transient-infinite  horizon  nodal 
described  in  Section  IV.  We  consider  the  problem  of  an  institution  that  wishes 
to  expand  its  faculty  size  by  500Z.  The  model  is  used  to  measure  the  tradeoff 
between  cost  and  the  length  of  the  growth  period.  We  checked  growth  periods 
from  8  to  24  years,  and  plotted  cost  vs.  growth  period  in  the  curve  shown  below. 

The  cost  figures  have  been  normalized  so  that  they  show  the  percent  of 
saving  that  is  achieved  by  lengthening  the  growth  period.  The  cost  calculated 
is  a  lower  bound  on  the  discounted  cost  of  going  from  an  initial  faculty 
distribution  to  a  target  distribution  (5  times  as  large)  in  T  years  and  then 
remaining  at  that  target  indefinitely. 

In  the  example  solved,  the  following  data  were  used: 


Initial  distribution:  (.3  .3  .A) 

Targst  distribution:  (.31  .275  .415) 

Targtr  sirs:  five  times  original  size-geometric  growth. 

Discount  factor:  a  ■  .96 
Support  costs:  (20  28  34) 

Appointment  costs:  (2  2  2) 

Proaotlon  matrix  in  ths  growth  stags 

.71 

Proaotlon  matrix  In  ths  steady  stats. 

.65  .08  0 
.8  .1 
.93 

Ths  growth  rats*  of  course,  Is  adjustsd  to  asst  ths  targst  slzs  in  ths  spsdfled 
tins  psrlod.  Ths  cosffldsnts  ars  soaswhat  arbitrary,  particularly  ths  discount 
rats.  Ths  sstlaatss  of  support  and  appolntasnt  cost,  and  ths  proaotlon  matrices, 
as  notad  abovs,  contain  ths  raault  of  policy  decisions.  Ths  small  size  of  this 
modal  allows  ona  to  test  sensitivity  to  changes  In  ths  discount  factor,  salary 
or  proaotlon  policy. 

Although  our  three  stats  model  Is  crude.  It  must  be  judged  against  past 
psrforaancs  of  faculty  planning  In  najor  universities.  Some  universities  have 
bean  through  expansion  periods  In  ths  post-war  era  without  considering  the  prob¬ 
lems  that  would  ensue  when  the  growth  period  was  over.  A  naive  shortsighted 
strategy  produced  a  nonequlllbrlua  result.  This  Is  akin  to  impacting  astronauts 
on  the  moon  rather  than  landing  them  there.  In  the  first  Instance  you  attain 
your  target  with  catastrophic  results.  In  the  second  case  a  smooth  transition 
to  equilibrium  is  assured.  Our  model  does  produce  a  smooth  transition  to 


equilibrium.  In  addition,  we  can  determine  a  leaat  coat  path  to  the  equilibrium 
distribution. 

Another  possible  use  of  the  model  is  in  determination  of  the  minimum  time 
needed  tc  reach  a  given  distribution.  For  this  problem  the  final  distribution 
x(T)  is  fixed  and  T  is  then  reduced  until  the  problem  becomes  infeasible. 

This  approach  not  only  yields  the  minimum  number  of  periods  until  x(T)  can 
be  reached  but  it  gives  a  tradeoff  curve  relating  the  number  of  periods  and  the 
cost  of  reaching  the  target.  If  the  model  is  of  reasonable  size  there  will  be 
little  computational  difficulty  in  this  approach,  and  a  decision  maker  could 
resolve  the  same  minimum  time  problem  under  various  assumptions  on  the  promotion 
matrix  to  gauge  the  effects  of  promotion  policy  on  the  time  to  reach  a  target. 
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